|
Author |
Thread Statistics | Show CCP posts - 32 post(s) |
|

CCP Explorer
C C P C C P Alliance
2198

|
Posted - 2014.05.29 09:56:00 -
[1] - Quote
The cause for this issue has been identified, we are working on the resolution. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2198

|
Posted - 2014.05.29 11:11:00 -
[2] - Quote
CCP Explorer wrote:The cause for this issue has been identified, we are working on the resolution. The exact problem has now been identified and we are working on a fix. Once implemented pilots will people will be able to complete their jobs normally. Stay tuned. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2198

|
Posted - 2014.05.29 11:16:00 -
[3] - Quote
Aluka 7th wrote:Just rollback yesterdays 14MB patch (->.789891) that is not mentioned on Patch Notes page...  The patch has no relevance to this issue. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2198

|
Posted - 2014.05.29 11:17:00 -
[4] - Quote
Steijn wrote:Aluka 7th wrote:Just rollback yesterdays 14MB patch that is not mentioned on Patch Notes page...  its fairly obvious that yesterdays patch included some code that made slots 'disappear'. If its linked to something that is getting deployed on the update next week and not before, could be an interesting few days. There is no connection between yesterday's patch and this issue.
Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2198

|
Posted - 2014.05.29 11:18:00 -
[5] - Quote
Silvetica Dian wrote:Jobs started yesterday before DT and the day before won't deliver. Jobs i started after the last DT deliver fine. Hope this distinction helps you narrow done the bug. Thanks for the details. We know what happened and working on the fix. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2198

|
Posted - 2014.05.29 11:18:00 -
[6] - Quote
Arline Kley wrote:CCP Explorer wrote:The cause for this issue has been identified, we are working on the resolution. Was it due to exploiting the workers by hanging on to outdated Federation dogma that perpetuates the economic and social differences in our society? Uhh, ummm, no. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2198

|
Posted - 2014.05.29 11:19:00 -
[7] - Quote
Loraine Gess wrote:Rivr Luzade wrote:I don't want to sound croaks of the fire-bellied toad, but I have the feeling that your 6 week cycle for patches is a tiny-itsy little bit too ambitious.  Not properly working patches, no patch notes, people not being able to download the patch, people not being able to log in... This has nothing to do with the release cycle?? Indeed, no connection. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2198

|
Posted - 2014.05.29 11:21:00 -
[8] - Quote
Tristan Darkheart wrote:CCP Explorer wrote:CCP Explorer wrote:The cause for this issue has been identified, we are working on the resolution. The exact problem has now been identified and we are working on a fix. Once implemented pilots will people will be able to complete their jobs normally. Stay tuned. So you will have a quick DT after you figure it out? Or are we looking at a full day? And are the jobs lost forever, or will they be okay? Targeted live testing of the fix 5 minutes ago was successful so no extended DT (the server is actually already up) or a reboot will be required. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2198

|
Posted - 2014.05.29 11:37:00 -
[9] - Quote
Apollo Purvon wrote:CCP Explorer wrote:Tristan Darkheart wrote:CCP Explorer wrote:CCP Explorer wrote:The cause for this issue has been identified, we are working on the resolution. The exact problem has now been identified and we are working on a fix. Once implemented pilots will people will be able to complete their jobs normally. Stay tuned. So you will have a quick DT after you figure it out? Or are we looking at a full day? And are the jobs lost forever, or will they be okay? Targeted live testing of the fix 5 minutes ago was successful so no extended DT (the server is actually already up) or a reboot will be required. Which server did you live test this on? It wasn't TQ because jobs on TQ are still broken. We only did *targeted* (limited) live testing. The full fix is now in review and pending execution. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2198

|
Posted - 2014.05.29 11:38:00 -
[10] - Quote
EMT Holding wrote:Any chance you could give some more detail on what went wrong and/or why? I'm curious. Yep, once we have fixed this. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|
|

CCP Explorer
C C P C C P Alliance
2198

|
Posted - 2014.05.29 11:43:00 -
[11] - Quote
Stevonix wrote:Well DT it come and DT it pass by and still I get the irritating 'Installation missing' message. Do we have to wait until DT tomorrow for a fix? My solicitors are demanding compensation for lost production CCP. Hey where's my missing T2 stuff? I filed a ticket too. We won't have to wait until DT tomorrow and we won't have to reboot. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2199

|
Posted - 2014.05.29 11:49:00 -
[12] - Quote
CCP Explorer wrote:Aluka 7th wrote:Just rollback yesterdays 14MB patch (->.789891) that is not mentioned on Patch Notes page...  The patch has no relevance to this issue. The patch notes have now been published. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2199

|
Posted - 2014.05.29 11:55:00 -
[13] - Quote
Ramman K'arojic wrote:So are we talking like in the next hour or like some before the EU work day ends ? The database script we are about to run will probably take about 1.5 hours. So we currently expect everything to be fixed around 13:30 EVE Time / GMT. Individual jobs will get fixed as the script executes. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2199

|
Posted - 2014.05.29 12:47:00 -
[14] - Quote
CCP Explorer wrote:Ramman K'arojic wrote:So are we talking like in the next hour or like some before the EU work day ends ? The database script we are about to run will probably take about 1.5 hours. So we currently expect everything to be fixed around 13:30 EVE Time / GMT. Individual jobs will get fixed as the script executes. The 1.5 hour estimate was based on the runtime of the activity that caused the issue. When the fix started executing it looked like it was going to run for 13.3 hours. We are now working on optimising the fix. It's already picking up speed and we'll keep you updated on the progress. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2199

|
Posted - 2014.05.29 12:58:00 -
[15] - Quote
CCP Explorer wrote:CCP Explorer wrote:Ramman K'arojic wrote:So are we talking like in the next hour or like some before the EU work day ends ? The database script we are about to run will probably take about 1.5 hours. So we currently expect everything to be fixed around 13:30 EVE Time / GMT. Individual jobs will get fixed as the script executes. The 1.5 hour estimate was based on the runtime of the activity that caused the issue. When the fix started executing it looked like it was going to run for 13.3 hours. We are now working on optimising the fix. It's already picking up speed and we'll keep you updated on the progress. Current estimate is a total runtime of 230 minutes, starting at 12:00 EVE Time / GMT.
Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2199

|
Posted - 2014.05.29 13:15:00 -
[16] - Quote
CCP Explorer wrote:CCP Explorer wrote:CCP Explorer wrote:Ramman K'arojic wrote:So are we talking like in the next hour or like some before the EU work day ends ? The database script we are about to run will probably take about 1.5 hours. So we currently expect everything to be fixed around 13:30 EVE Time / GMT. Individual jobs will get fixed as the script executes. The 1.5 hour estimate was based on the runtime of the activity that caused the issue. When the fix started executing it looked like it was going to run for 13.3 hours. We are now working on optimising the fix. It's already picking up speed and we'll keep you updated on the progress. Current estimate is a total runtime of 230 minutes, starting at 12:00 EVE Time / GMT. We are now estimating to be completely done at 14:56 EVE Time / GMT (total runtime 176 minutes). The fix is incremental so jobs are being fixed one at a time. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2199

|
Posted - 2014.05.29 13:30:00 -
[17] - Quote
Maciff Hellenow Mernher wrote:Procrastination completely ruined -.-
Now what todo...... I'm going to have some coffee for the next hour or so. Then come back here. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2199

|
Posted - 2014.05.29 13:37:00 -
[18] - Quote
CCP Explorer wrote:CCP Explorer wrote:CCP Explorer wrote:CCP Explorer wrote:Ramman K'arojic wrote:So are we talking like in the next hour or like some before the EU work day ends ? The database script we are about to run will probably take about 1.5 hours. So we currently expect everything to be fixed around 13:30 EVE Time / GMT. Individual jobs will get fixed as the script executes. The 1.5 hour estimate was based on the runtime of the activity that caused the issue. When the fix started executing it looked like it was going to run for 13.3 hours. We are now working on optimising the fix. It's already picking up speed and we'll keep you updated on the progress. Current estimate is a total runtime of 230 minutes, starting at 12:00 EVE Time / GMT. We are now estimating to be completely done at 14:56 EVE Time / GMT (total runtime 176 minutes). The fix is incremental so jobs are being fixed one at a time. 14:45 (runtime 165 minutes) is the latest estimate. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2200

|
Posted - 2014.05.29 14:36:00 -
[19] - Quote
CCP Explorer wrote:CCP Explorer wrote:CCP Explorer wrote:CCP Explorer wrote:CCP Explorer wrote:Ramman K'arojic wrote:So are we talking like in the next hour or like some before the EU work day ends ? The database script we are about to run will probably take about 1.5 hours. So we currently expect everything to be fixed around 13:30 EVE Time / GMT. Individual jobs will get fixed as the script executes. The 1.5 hour estimate was based on the runtime of the activity that caused the issue. When the fix started executing it looked like it was going to run for 13.3 hours. We are now working on optimising the fix. It's already picking up speed and we'll keep you updated on the progress. Current estimate is a total runtime of 230 minutes, starting at 12:00 EVE Time / GMT. We are now estimating to be completely done at 14:56 EVE Time / GMT (total runtime 176 minutes). The fix is incremental so jobs are being fixed one at a time. 14:45 (runtime 165 minutes) is the latest estimate. 14:40 (runtime 160 minutes), or in completely done in 4 minutes. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2200

|
Posted - 2014.05.29 14:41:00 -
[20] - Quote
CCP Explorer wrote:CCP Explorer wrote:CCP Explorer wrote:CCP Explorer wrote:CCP Explorer wrote:CCP Explorer wrote:Ramman K'arojic wrote:So are we talking like in the next hour or like some before the EU work day ends ? The database script we are about to run will probably take about 1.5 hours. So we currently expect everything to be fixed around 13:30 EVE Time / GMT. Individual jobs will get fixed as the script executes. The 1.5 hour estimate was based on the runtime of the activity that caused the issue. When the fix started executing it looked like it was going to run for 13.3 hours. We are now working on optimising the fix. It's already picking up speed and we'll keep you updated on the progress. Current estimate is a total runtime of 230 minutes, starting at 12:00 EVE Time / GMT. We are now estimating to be completely done at 14:56 EVE Time / GMT (total runtime 176 minutes). The fix is incremental so jobs are being fixed one at a time. 14:45 (runtime 165 minutes) is the latest estimate. 14:40 (runtime 160 minutes), or in completely done in 4 minutes. The fix was fully applied at 14:40 EVE Time / GMT.
Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|
|

CCP Explorer
C C P C C P Alliance
2201

|
Posted - 2014.05.29 15:23:00 -
[21] - Quote
Jaiimez Skor wrote:CCP Explorer wrote:Arline Kley wrote:CCP Explorer wrote:The cause for this issue has been identified, we are working on the resolution. Was it due to exploiting the workers by hanging on to outdated Federation dogma that perpetuates the economic and social differences in our society? Uhh, ummm, no. Oh bless that Monty Python reference has gone straight over you head hasn't it Explorer. It did. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2204

|
Posted - 2014.05.29 21:22:00 -
[22] - Quote
Querns wrote:Dinsdale Pirannha wrote:I look forward to all the other mechanics you will wreck by ramming through massive changes without proper testing. If you guys seriously believe that changes of this magnitude and complexity, larger than Apochrypha, are going to be all sorted with at most 6 weeks of testing on Sisi, you are seriously deluded.
CCP, use whatever common sense you have left, stop listening to your null sec cartel "advisors", and postpone this disaster-in-waiting until the fall, and allow for proper testing on Sisi. What proof do you have that the issue was even related to the pending industry changes? I don't think he has any, but this was industry-related pre-cleanup. Testing of the industry changes begins on Singularity around 10 June and the release is currently scheduled for 22 July. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2204

|
Posted - 2014.05.29 21:23:00 -
[23] - Quote
Bjurn Akely wrote:CCP Explorer wrote:Jaiimez Skor wrote:CCP Explorer wrote:Arline Kley wrote:CCP Explorer wrote:The cause for this issue has been identified, we are working on the resolution. Was it due to exploiting the workers by hanging on to outdated Federation dogma that perpetuates the economic and social differences in our society? Uhh, ummm, no. Oh bless that Monty Python reference has gone straight over you head hasn't it Explorer. It did. You make me sad. So be it. I hang my head in shame...
Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2204

|
Posted - 2014.05.29 23:48:00 -
[24] - Quote
asteroidjas wrote:CCP Explorer wrote:EMT Holding wrote:Any chance you could give some more detail on what went wrong and/or why? I'm curious. Yep, once we have fixed this. So.....where is the info on what went wrong? I've just checked every page of this thread and looked for any actual info...since you claim it had nothing to do with the stealth patch from yesterday... Right now pending that I have an opportunity to talk to the team involved. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2204

|
Posted - 2014.05.30 08:18:00 -
[25] - Quote
Dealth Striker wrote:If CCP knew what the problem was and they fixed it. Why did Explorer dodge the question about what happened? I think it is so they can polish a response. See my reply above, so I can talk to the team.
Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2204

|
Posted - 2014.05.30 20:56:00 -
[26] - Quote
CCP Explorer wrote:asteroidjas wrote:CCP Explorer wrote:EMT Holding wrote:Any chance you could give some more detail on what went wrong and/or why? I'm curious. Yep, once we have fixed this. So.....where is the info on what went wrong? I've just checked every page of this thread and looked for any actual info...since you claim it had nothing to do with the stealth patch from yesterday... Right now pending that I have an opportunity to talk to the team involved. And I wasn't able to get to this today, will have to wait until after the weekend. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2204

|
Posted - 2014.05.31 11:27:00 -
[27] - Quote
Don Aubaris wrote:CCP Explorer wrote: ... And I wasn't able to get to this today, will have to wait until after the weekend. Let me guess what happend : Dev1 gets an SMS. Dev1 : "Quick. CCP Explorer is on his way to here. We have to move again." Dev2 : "Where to now?" Dev1 : "The basement. I still have some bottles stashed away there from FanFest." Dev2 : "We never should have run that script without testing" Dev1 : "Testing. Testing. Testing. Testing is for sissies. Live on the edge, man. And now move" Dev2 : "He is gonna find us eventually". Dev1 : 'Nah. it's almost weekend. And against next week someone else will have screwed up something and all this will all be forgotten" Dev1 and Dev2 leave through door 1. CCP Explorer enters through door 2, looks around and scratches his head. This is uncanny, how do you know this?!? Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2212

|
Posted - 2014.06.03 08:54:00 -
[28] - Quote
So, the details:
When blueprints are used for science & industry jobs, we move them into a special location so that they are not accessible while being used. After the job completes then we return the blueprint (if it's an original or a copy with remaining runs) or put into a junkyard (copy with no remaining runs).
There is an old bug in the system that causes blueprints to remain in that special location when they should have been moved into a junkyard. There are about 41 million blueprints there now. The aim of this cleanup was to find all those blueprints and move them. The script had an error causing some additional blueprint copies that were a part of a current job to be moved as well.
Through a series of unfortunate miscommunication the cleanup was run by system administrators in Shanghai when the dev team in Reykjav+¡k was on a public holiday, which explains the delayed response. Once the issue was discovered we reverted the entire cleanup. Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|

CCP Explorer
C C P C C P Alliance
2213

|
Posted - 2014.06.03 12:14:00 -
[29] - Quote
Salpun wrote:Thanks for the details. When will this script be ran again? Not yet decided.
Erlendur S. Thorsteinsson | Senior Development Director | EVE Online // CCP Games | @erlendur |
|
|
|
|